2024 The Mathematics of Reliable Artificial Intelligence.Kutyniok

GPT-4 等基础模型最近取得了前所未有的成功，提高了公众对人工智能 (AI) 的认识，并引发了对其相关可能性和威胁的热烈讨论。2023 年 3 月，一群技术领袖发表了一封公开信，呼吁公众暂停人工智能开发，以便有时间创建和实施共享安全协议。世界各地的政策制定者还通过各种监管措施应对人工智能技术的快速发展，包括欧盟(EU) 人工智能法案和广岛人工智能进程。

The recent unprecedented success of foundation models like GPT-4 has heightened the general public’s awareness of artificial intelligence (AI) and inspired vivid discussion about its associated possibilities and threats. In March 2023, a group of technology leaders published an open letter that called for a public pause in AI development to allow time for the creation and implementation of shared safety protocols. Policymakers around the world have also responded to rapid advancements in AI technology with various regulatory efforts, including the European Union (EU) AI Act and the Hiroshima AI Process.

人工智能技术目前面临的问题之一（以及由此产生的危险）是其不可靠性和随之而来的不可信性。近年来，基于人工智能的技术在安全性、保密性、隐私以及公平性和可解释性方面的责任方面经常遇到严重问题。隐私侵犯、不公平的决策、无法解释的结果以及涉及自动驾驶汽车的事故都是令人担忧的结果的例子。

One of the current problems—and consequential dangers—of AI technology is its unreliability and subsequent lack of trustworthiness. In recent years, AI-based technologies have often encountered severe issues in terms of safety, security, privacy, and responsibility with respect to fairness and interpretability. Privacy violations, unfair decisions, unexplainable results, and accidents involving self-driving cars are all examples of concerning outcomes.

克服这些和其他问题，同时满足法律要求，需要深刻的数学理解。在这里，我们将探索可靠人工智能的数学 [1]，特别关注人工神经网络：人工智能当前的主力。人工神经网络并不是一个新现象；1943 年，沃伦·麦卡洛克和沃尔特·皮茨通过引入数学模型来模仿人类大脑的功能，开发了初步的学习算法方法，该模型由神经元网络组成 [10]。他们的方法启发了以下人工神经元的定义：

Overcoming these and other problems while simultaneously fulfilling legal requirements necessitates a deep mathematical understanding. Here, we will explore the mathematics of reliable AI [1] with a particular focus on artificial neural networks: AI’s current workhorse. Artificial neural networks are not a new phenomenon; in 1943, Warren McCulloch and Walter Pitts developed preliminary algorithmic approaches to learning by introducing a mathematical model to mimic the functionality of the human brain, which consists of a network of neurons [10]. Their approach inspired the following definition of an artificial neuron:

$f （十_{1} ，。。。，十_{n} ） = ρ （ \sum_{杰 = 1}^{n} 十_{杰} 瓦_{杰} - b ）， f ： R^{n} \to R ，$

$f (x_{1}, . . ., x_{n}) = ρ (\sum_{j = 1}^{n} x_{j} w_{j} - b), f : R^{n} \to R,$

带重量 $瓦_{1} ，。。。，瓦_{n} \in R$ ，偏见 $b \in R$ 以及激活函数 $ρ ： R \to R$ ，通常是修正线性单元 (ReLU) $ρ （十） = 最大限度 {0 ，十}$ 将这些人工神经元排列成层，就得到了人工神经网络的定义 $φ$ 深度 $大号$ ：

with weights $w_{1}, . . ., w_{n} \in R$ , bias $b \in R$ , and activation function $ρ : R \to R$ , which is typically the rectified linear unit (ReLU) $ρ (x) = max {0, x}$ . Arranging these artificial neurons into layers yields the definition of an artificial neural network $Φ$ of depth $L$ :

$φ （十）：= {电视}_{大号} ρ （ {电视}_{大号 - 1} ρ （ \dots （ ρ （ {电视}_{1} （十））） \dots ））， φ ： R^{n} \to R ，$

$Φ (x) := T_{L} ρ (T_{L - 1} ρ (\dots (ρ (T_{1} (x))) \dots)), Φ : R^{n} \to R,$

在哪里 ${电视}_{钾} （十） = 西_{钾} （十） - b_{钾}$ ， $钾 = 1 ， \dots ，大号$ 是仿射线性函数 $西_{钾}$ 作为权重矩阵和 $b_{钾}$ 作为偏差向量。值得注意的是，人工神经网络确实是纯粹的数学对象。

where $T_{k} (x) = W_{k} (x) - b_{k}$ , $k = 1, \dots, L$ are affine-linear functions with $W_{k}$ as the weight matrices and $b_{k}$ as the bias vectors. It is worth noting that an artificial neural network is indeed a purely mathematical object.

接下来，我们来讨论一下（人工）神经网络的应用工作流程，这将自动引出可靠性领域的关键研究方向。给定一个数据集 $（十_{我} ，是_{我} ）_{我}$ — 可以从流形上的数据分类函数中采样 $米$ ， IE， $克：米 \to {1 ，。。。，钾}$ —神经网络的关键任务 $φ$ 是近似数据和底层函数 $克$ 将数据集拆分为训练集和测试集后，我们可以进行如下操作：

Let us next discuss the workflow for the application of (artificial) neural networks, which automatically leads to key research directions in the realm of reliability. Given a data set $(x_{i}, y_{i})_{i}$ —which may be sampled from a classification function of data on a manifold $M$ , i.e., $g : M \to {1, . . ., K}$ —the key task of a neural network $Φ$ is to approximate the data and the underlying function $g$ . After splitting the data set into a training set and a test set, we can proceed as follows:

（i）通过确定网络中的层数、每层的神经元数量等来选择架构。

(i) Choose an architecture by determining the number of layers in the network, the number of neurons in each layer, and so forth.

（ii）通过优化权重矩阵和偏差向量来训练神经网络。此步骤通过随机梯度下降完成，解决了优化问题

(ii) Train the neural network by optimizing the weight matrices and bias vectors. This step is accomplished via stochastic gradient descent, which solves the optimization problem

$\underset{西_{钾} ， b_{钾}}{分钟} \sum_{我} 大号（ φ_{西_{钾} ， b_{钾}} （十_{我} ），是_{我} ） + λ R （西_{钾} ， b_{钾} ）。$

$min_{W_{k}, b_{k}} \sum_{i} L (Φ_{W_{k}, b_{k}} (x_{i}), y_{i}) + λ R (W_{k}, b_{k}) .$

这里， $大号$ 是损失函数（例如平方损失）， $R$ 是一个正则化项。

Here, $L$ is a loss function (such as the square loss) and $R$ is a regularization term.

（iii）使用测试数据分析训练后的神经网络对未知数据进行推广的能力。

(iii) Use the test data to analyze the trained neural network’s ability to generalize to unseen data.

这些步骤导致了三个特定的研究方向——表现力、训练和泛化——与统计学习问题中的三个错误成分相关：近似误差、算法误差和样本外误差。

These steps lead to three particular research directions—expressivity, training, and generalization—that are associated with the three error components in a statistical learning problem: approximation error, error from the algorithm, and out-of-sample error.

<strong>图 1.</strong> 双下降曲线展示了过度参数化的积极影响。图片由作者提供。 — **图 1.**双下降曲线展示了过度参数化的积极影响。图片由作者提供。

表达力领域旨在确定所考虑的神经网络类别相对于某些“自然”函数类别的近似属性，同时通常还考虑神经网络在非零参数（权重和偏差）数量方面的所需复杂性。表达力可能是人工智能数学研究方向中探索最彻底的。早期的亮点是 20 世纪 80 年代著名的通用近似定理 [6] 的发展。该定理表明，对于浅层神经网络的非多项式激活函数，我们可以将任何连续函数近似到任意程度，这在当时是最先进的。有趣的是，最近的结果证明神经网络可以模拟大多数已知的近似方案，包括通过小波和剪切波等仿射系统进行的近似 [4]。

The area of expressivity seeks to determine approximation properties of the considered class of neural networks with respect to certain “natural” function classes, while also typically accounting for the neural networks’ required complexity in terms of the number of nonzero parameters (weights and biases). Expressivity is perhaps the most thoroughly explored mathematical research direction of AI. An early highlight was the development of the famous universal approximation theorem in the 1980s [6]. This theorem shows that we can approximate any continuous function up to an arbitrary degree for non-polynomial activation functions of shallow neural networks, which were state of the art at the time. Intriguingly, recent results prove that neural networks can simulate most known approximation schemes, including approximation by affine systems like wavelets and shearlets [4].

训练的难度源于优化问题的高度非凸性以及损失图中伪局部最小值、鞍点和局部最大值的存在。因此，随机梯度下降似乎在最终的泛化性能中找到“良好”的局部最小值，这尤其令人惊讶。要实现这一成功，需要通过结合训练动态和代数几何等领域的技术来分析损失图景。最近的一个亮点是神经崩溃现象[11]，其中类特征在训练的最后阶段在特征空间中形成分离良好的簇。这个结果帮助我们理解了为什么训练误差超过零点不会产生高度过拟合的模型，正如我们预期的那样。

The difficulty of training stems from the optimization problem’s high nonconvexity and the presence of spurious local minima, saddle points, and local maxima in the loss landscape. It is therefore particularly surprising that stochastic gradient descent seems to find “good” local minima in the resulting generalization performance. Achieving this success requires analysis of the loss landscape via a combination of training dynamics and techniques from areas such as algebraic geometry. One recent highlight is the phenomenon of neural collapse [11], wherein the class features form well-separated clusters in feature space during the final stages of training. This result helped us understand why training beyond the point of zero training error does not yield a highly overfitted model, as we might expect.

我们基本上可以将泛化细分为两类：函数分析方法和随机/统计方法。第一类方法通常旨在确定性设置中的误差界限。例如，我们可以精确确定谱图卷积神经网络对于模拟相同现象的输入图的泛化误差，例如图元意义上的泛化误差 [9]。相比之下，第二类方法通常试图分析所谓的双下降曲线，它表现出过度参数化的惊人积极效果（见图 1）。这种分析通常依赖于 Vapnik-Chervonenkis 维度、Rademacher 复杂度或神经正切核等方法 [7]。

We can essentially subdivide generalization into two classes: functional analytic approaches and stochastic/statistical approaches. The first class typically aims for error bounds in deterministic settings. For example, we can precisely determine the generalization error of spectral graph convolutional neural networks for input graphs that model the same phenomenon, e.g., in the sense of graphons [9]. In contrast, the second class typically seeks to analyze the so-called double descent curve, which exhibits the surprisingly positive effects of overparameterization (see Figure 1). Such analysis usually relies on methods like the Vapnik-Chervonenkis dimension, Rademacher complexity, or neural tangent kernels [7].

对表达力、训练和泛化的深刻数学理解对于确保可靠性至关重要。然而，上述欧盟人工智能法案和其他类似法规质疑，如果没有关于整个训练过程的详细信息，可靠性是否可行。这些政策要求人工智能技术拥有“解释权”。这些要求导致了可解释性概念的产生，该概念旨在通过确定和突出显示有助于特定决策的输入数据的主要特征来阐明神经网络做出决策的方式。这种能力对于向客户解释决策以及从科学应用中的数据中获取更多见解都非常有用。一个目标是开发可解释性方法，使人类用户能够以与人类交流的方式与神经网络交流；大型语言模型的出现使这一愿景更接近现实。但从数学的角度来看，这种方法本身也必须是可靠的。目前有几种潜在的基于数学的可解释性方法，例如博弈论中的 Shapley 值 [13] 和信息论中的速率失真解释 [8]。

A deep mathematical understanding of expressivity, training, and generalization will be crucial to ensure reliability. However, the aforementioned EU AI Act and other similar regulations question whether reliability is achievable without detailed information about the entire training process. These policies ask for a “right to explanation” for AI technologies. Such requests lead to the concept of explainability, which aims to clarify the way in which neural networks reach decisions by determining and highlighting the main features of the input data that contribute to a particular decision. This ability would be highly useful for both explaining decisions to customers and deriving additional insights from data in scientific applications. One goal is to develop explainability approaches that enable human users to communicate with a neural network in the same way that they might communicate with a human; the advent of large language models has brought this vision one step closer to reality. But from a mathematical standpoint, such an approach must also be reliable itself. Several potential mathematically grounded explainability methods are presently available, such as Shapley values from game theory [13] and rate-distortion explanations from information theory [8].

<strong>图 2.</strong> 受半导体研究公司的半导体十年计划启发，能源生产和消耗随时间的变化 [12]。图片由作者提供。 — **图 2.**受半导体研究公司半导体十年计划启发，能源生产和消耗随时间的变化情况 [12]。图片由作者提供。

虽然用户目前正在将深度神经网络和 AI 技术应用于科学和工业领域的各种问题，但这些方法确实存在很大的局限性。不幸的是，这一研究方向目前并不是主要关注点，但一些结果仍然值得强调。例如，最近的研究表明，保证任何学习问题达到给定均匀准确度所需的最小训练样本数量在网络架构的深度和输入维度上都呈指数级增长；这意味着学习 ReLU 网络以达到高均匀准确度是难以解决的 [2]。2022 年，另一项研究分析了在数字硬件（如图形处理单元）上运行基于 AI 的算法的问题，该硬件被建模为图灵机，而问题本身通常具有连续性 [5]。不幸的是，这种差异使得各种问题（包括逆问题）无法计算，并导致严重的可靠性问题。同时，其他结果表明，与创新模拟硬件（如神经形态芯片或量子计算）相关的 Blum-Shub-Smale 机器可以克服这一障碍 [3]。这样的硬件有望克服人们对数字硬件能耗的严重担忧（见图2），这是美国《芯片与科学法案》中的一项关键内容。

While users are currently applying deep neural networks and AI techniques to a wide variety of problems in science and industry, these methods do have significant limitations. This research direction is unfortunately not a major focus at the moment, but some results are still worth highlighting. For example, recent work demonstrated that the minimal number of training samples to guarantee a given uniform accuracy on any learning problem scales exponentially in both the depth and input dimension of the network architecture; this means that learning ReLU networks to high uniform accuracy is intractable [2]. In 2022, another study analyzed the problem of running AI-based algorithms on digital hardware (like graphical processing units) that is modeled as a Turing machine, whereas the problems themselves are typically of a continuum nature [5]. Unfortunately, this discrepancy makes various problems—including inverse problems—noncomputable and causes serious reliability issues. At the same time, other results indicate that Blum-Shub-Smale machines—which relate to innovative analog hardware, such as neuromorphic chips or quantum computing—could surmount this obstacle [3]. Such hardware will hopefully also overcome the acute concern of energy consumption by digital hardware (see Figure 2), which is a key item in the U.S. CHIPS and Science Act.

总而言之，不可靠性是人工智能技术发展中最严重的障碍之一，而数学的许多领域将有助于解决这一复杂问题。此外，只有通过将“解释权”等术语数学化，才能实现对基于人工智能的方法在法律上要求的属性的自动验证。因此，人工智能的可靠性与数学密不可分，最终为我们的社区创造了非常令人兴奋的研究机会。

To summarize, unreliability is one of the most serious impediments in the development of AI technology, and many areas of mathematics will help address this complication. Furthermore, the automatic verification of properties that are legally required for AI-based approaches is only attainable through a mathematization of terms like the “right to explanation.” AI reliability is hence inextricably linked to mathematics, ultimately creating very exciting research opportunities for our community.

参考

[1] Berner, J.、Grohs, P.、Kutyniok, G. 和 Petersen, P. (2022)。深度学习的现代数学。在 P. Grohs 和 G. Kutyniok (Eds.) 的《 深度学习的数学方面》（第 1-111 页）中。英国剑桥：剑桥大学出版社。
[2] Berner, J.、Grohs, P. 和 Voigtlaender, F. (2023)。将 ReLU 网络学习到高均匀精度是难以解决的。在第 11 届国际学习表征会议 (ICLR 2023)上。卢旺达基加利。
[3] Boche, H.、Fono, A. 和 Kutyniok, G. (2022)。逆问题可以在实数信号处理硬件上解决。预印本，arXiv:2204.02066。
[4] Bölcskei, H.、Grohs, P.、Kutyniok, G. 和 Petersen, P. (2019)。稀疏连接深度神经网络的最佳近似。SIAM J. Math. Data Sci.，1 (1)，8-45。
[5] Colbrook, MJ、Antun, V. 和 Hansen, AC (2022)。计算稳定和准确的神经网络的难度：论深度学习的障碍和 Smale 的第 18 个问题。美国国家科学院院刊，119 (12)，e2107151119。
[6] Cybenko, G. (1989)。通过叠加 S 型函数进行近似。数学控制信号系统，2，303-314。
[7] Jacot, A.、Gabriel, F. 和 Hongler, C. (2018)。神经切线核：神经网络中的收敛和泛化。在NIPS'18：第 32 届神经信息处理系统国际会议论文集（第 8580-8589 页）中。加拿大蒙特利尔：Curran Associates, Inc.
[8] Kolek, S.、Nguyen, DA、Levie, R.、Bruna, J. 和 Kutyniok, G. (2022)。用于解释黑盒模型决策的速率失真框架。在 A. Holzinger、R. Goebel、R. Fong、T. Moon、K.-R. Müller 和 W. Samek (Eds.) 的xxAI - 超越可解释的 AI（第 91-115 页）中。计算机科学讲义（第 13200 卷）。瑞士 Cham：Springer。
[9] Levie, R.、Huang, W.、Bucci, L.、Bronstein, M. 和 Kutyniok, G. (2021)。谱图卷积神经网络的可迁移性。J . Mach. Learn. Res.，22 (1)，12462-12520。
[10] McCulloch, WS 和 Pitts, W. (1943)。神经活动中内在思想的逻辑演算。数学生物物理学公报，5，115-133。
[11] Papyan, V.、Han, XY 和 Donoho, DL (2020)。深度学习训练末期神经崩溃的普遍性。美国国家科学院院刊，117 (40)，24652-24663。
[12] Semiconductor Research Corporation (2021)。半导体十年计划：完整报告。北卡罗来纳州达勒姆：半导体研究公司。检索自https://www.src.org/about/decadal-plan。[13 ]
Štrumbelj, E.，& Kononenko, I. (2010)。使用博弈论对个体分类进行有效解释。J . Mach. Learn. Res.，11（1），1-18。

References

[1] Berner, J., Grohs, P., Kutyniok, G., & Petersen, P. (2022). The modern mathematics of deep learning. In P. Grohs & G. Kutyniok (Eds.), Mathematical aspects of deep learning (pp. 1-111). Cambridge, U.K.: Cambridge University Press.
[2] Berner, J., Grohs, P., & Voigtlaender, F. (2023). Learning ReLU networks to high uniform accuracy is intractable. In The eleventh international conference on learning representations (ICLR 2023). Kigali, Rwanda.
[3] Boche, H., Fono, A., & Kutyniok, G. (2022). Inverse problems are solvable on real number signal processing hardware. Preprint, arXiv:2204.02066.
[4] Bölcskei, H., Grohs, P., Kutyniok, G., & Petersen, P. (2019). Optimal approximation with sparsely connected deep neural networks. SIAM J. Math. Data Sci., 1(1), 8-45.
[5] Colbrook, M.J., Antun, V., & Hansen, A.C. (2022). The difficulty of computing stable and accurate neural networks: On the barriers of deep learning and Smale’s 18th problem. Proc. Natl. Acad. Sci., 119(12), e2107151119.
[6] Cybenko, G. (1989). Approximation by superpositions of a sigmoidal function. Math. Control Signals Syst., 2, 303-314.
[7] Jacot, A., Gabriel, F., & Hongler, C. (2018). Neural tangent kernel: Convergence and generalization in neural networks. In NIPS’18: Proceedings of the 32nd international conference on neural information processing systems (pp. 8580-8589). Montreal, Canada: Curran Associates, Inc.
[8] Kolek, S., Nguyen, D.A., Levie, R., Bruna, J., & Kutyniok, G. (2022). A rate-distortion framework for explaining black-box model decisions. In A. Holzinger, R. Goebel, R. Fong, T. Moon, K.-R. Müller, & W. Samek (Eds.), xxAI - Beyond explainable AI (pp. 91-115). Lecture notes in computer science (Vol. 13200). Cham, Switzerland: Springer.
[9] Levie, R., Huang, W., Bucci, L., Bronstein, M., & Kutyniok, G. (2021). Transferability of spectral graph convolutional neural networks. J. Mach. Learn. Res., 22(1), 12462-12520.
[10] McCulloch, W.S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. Bull. Math. Biophys., 5, 115-133.
[11] Papyan, V., Han, X.Y., & Donoho, D.L. (2020). Prevalence of neural collapse during the terminal phase of deep learning training. Proc. Natl. Acad. Sci., 117(40), 24652-24663.
[12] Semiconductor Research Corporation (2021). Decadal plan for semiconductors: Full report. Durham, NC: Semiconductor Research Corporation. Retrieved from https://www.src.org/about/decadal-plan.
[13] Štrumbelj, E., & Kononenko, I. (2010). An efficient explanation of individual classifications using game theory. J. Mach. Learn. Res., 11(1), 1-18.